{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "itFnv18OcKMr"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/drive/1aGZv3bNDvrEOOCq78lSYREGiO7jpTgt7?usp=sharing\" target=\"_parent\">\n",
        "<img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"></a>\n",
        "\n",
        "#### [Huggingface Models](https://huggingface.co/) - Hugging Face models – Requires T4 GPU. Select ‘Change Runtime’ in Colab to enable."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "RyuBwY0SbE6a"
      },
      "source": [
        "### Install required libraries"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "L_USm-WLbCqx",
        "outputId": "b7370c96-bda9-4762-a848-1e6c635e1f68"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m10.4/10.4 MB\u001b[0m \u001b[31m34.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m76.1/76.1 MB\u001b[0m \u001b[31m9.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m354.7/354.7 kB\u001b[0m \u001b[31m10.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m363.4/363.4 MB\u001b[0m \u001b[31m4.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m13.8/13.8 MB\u001b[0m \u001b[31m35.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m24.6/24.6 MB\u001b[0m \u001b[31m29.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m883.7/883.7 kB\u001b[0m \u001b[31m35.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m664.8/664.8 MB\u001b[0m \u001b[31m2.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m211.5/211.5 MB\u001b[0m \u001b[31m5.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m56.3/56.3 MB\u001b[0m \u001b[31m12.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m127.9/127.9 MB\u001b[0m \u001b[31m7.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m207.5/207.5 MB\u001b[0m \u001b[31m5.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m21.1/21.1 MB\u001b[0m \u001b[31m92.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25h"
          ]
        }
      ],
      "source": [
        "!pip install --upgrade --quiet  langchain-huggingface text-generation transformers langchainhub bitsandbytes accelerate"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5f56hJjCbIsZ"
      },
      "source": [
        "### Import related libraries"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 2,
      "metadata": {
        "id": "-kKJmfwfbJN1"
      },
      "outputs": [],
      "source": [
        "from langchain_huggingface import ChatHuggingFace, HuggingFaceEndpoint"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 3,
      "metadata": {
        "id": "KRNrCaKLbKwP"
      },
      "outputs": [],
      "source": [
        "import os\n",
        "import getpass"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "1mR3usGWeNnw"
      },
      "source": [
        "### 🔑 Provide Huggingface API Key\n",
        "\n",
        "1. [Huggingface API Key](https://huggingface.co/docs/hub/security-tokens)\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 4,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "k4LH5qIAbP77",
        "outputId": "adfc8272-bbcb-4437-cba5-eb438385d728"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "··········\n"
          ]
        }
      ],
      "source": [
        "os.environ[\"HUGGINGFACEHUB_API_TOKEN\"] = getpass.getpass()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 5,
      "metadata": {
        "id": "v1hnl1F3bTwY"
      },
      "outputs": [],
      "source": [
        "from transformers import BitsAndBytesConfig\n",
        "\n",
        "quantization_config = BitsAndBytesConfig(\n",
        "    load_in_4bit=True,\n",
        "    bnb_4bit_quant_type=\"nf4\",\n",
        "    bnb_4bit_compute_dtype=\"float16\",\n",
        "    bnb_4bit_use_double_quant=True,\n",
        ")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Q4_SFw1qbUbK"
      },
      "outputs": [],
      "source": [
        "from langchain_huggingface import ChatHuggingFace, HuggingFacePipeline\n",
        "\n",
        "llm = HuggingFacePipeline.from_model_id(\n",
        "    model_id=\"deepseek-ai/DeepSeek-R1-Distill-Llama-8B\",\n",
        "    task=\"text-generation\",\n",
        "    pipeline_kwargs=dict(\n",
        "        max_new_tokens=512,\n",
        "        do_sample=False,\n",
        "        repetition_penalty=1.03,\n",
        "        return_full_text=False,\n",
        "    ),\n",
        "    model_kwargs={\"quantization_config\": quantization_config},\n",
        ")\n",
        "\n",
        "chat_model = ChatHuggingFace(llm=llm)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 7,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "XV2b9OhGbXSM",
        "outputId": "810a7408-93a9-4972-83f6-b4dc8baa7a62"
      },
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.11/dist-packages/transformers/generation/configuration_utils.py:631: UserWarning: `do_sample` is set to `False`. However, `temperature` is set to `0.6` -- this flag is only used in sample-based generation modes. You should set `do_sample=True` or unset `temperature`.\n",
            "  warnings.warn(\n",
            "/usr/local/lib/python3.11/dist-packages/transformers/generation/configuration_utils.py:636: UserWarning: `do_sample` is set to `False`. However, `top_p` is set to `0.95` -- this flag is only used in sample-based generation modes. You should set `do_sample=True` or unset `top_p`.\n",
            "  warnings.warn(\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Okay, so I'm trying to figure out what happens when an unstoppable force meets an immovable object. Hmm, this is an interesting question. Let me break it down.\n",
            "\n",
            "First, I need to understand what each term means. An \"unstoppable force\" is something that can't be stopped or prevented, right? Like a natural disaster or a powerful entity with immense power. On the other hand, an \"immovable object\" is something that can't be moved or changed, like a mountain or a very strong structure.\n",
            "\n",
            "So, if I imagine these two concepts colliding, what would happen? Well, if the force is unstoppable, it's going to try to move or change the object. But the object is immovable, meaning it won't budge. So, the force might try to push, pull, or break it, but the object remains in place.\n",
            "\n",
            "I wonder if the force could eventually overcome the object. Maybe if the force is strong enough or persistent enough, it could cause some damage or even destroy the object. But then again, the object is immovable, so maybe it's designed to withstand anything. It depends on the nature of both the force and the object.\n",
            "\n",
            "Wait, but what if the force isn't physical? Like, maybe it's a powerful wave of energy or something non-physical. In that case, could it just pass through or around the object? Or would it still try to interact in some way?\n",
            "\n",
            "Also, I should consider the context. In physics, an unstoppable force might be something like a black hole's gravitational pull, which could potentially \"swallow\" the object. But a black hole is more of a place rather than a force, so maybe that's not the best example.\n",
            "\n",
            "Another angle: in literature or movies, when an unstoppable force meets an immovable object, sometimes the result is destruction or a stalemate. For example, in some stories, the force might reshape the object or find a way to move it indirectly.\n",
            "\n",
            "I should also think about real-world examples. Think of a flood (force) versus a mountain (object). The flood could erode the mountain over time, but if the mountain is made of something very strong, like rock, the flood might just carve a path through it or around it. Alternatively, if the force is a tidal wave, it might cause significant damage but not necessarily move the mountain.\n",
            "\n",
            "Wait, but if the force is something like a tornado, it could uproot trees and move objects, but an immovable object like a\n"
          ]
        }
      ],
      "source": [
        "from langchain_core.messages import (\n",
        "    HumanMessage,\n",
        "    SystemMessage,\n",
        ")\n",
        "\n",
        "messages = [\n",
        "    SystemMessage(content=\"You're a helpful assistant\"),\n",
        "    HumanMessage(\n",
        "        content=\"What happens when an unstoppable force meets an immovable object?\"\n",
        "    ),\n",
        "]\n",
        "\n",
        "ai_msg = chat_model.invoke(messages)\n",
        "print(ai_msg.content)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "ODACef9ceTS1"
      },
      "outputs": [],
      "source": []
    }
  ],
  "metadata": {
    "accelerator": "GPU",
    "colab": {
      "gpuType": "T4",
      "provenance": []
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}
